Language and Speech Processing Final Project PCFG Parser

The use of statistical methods in areas such as language processing, speech recognition and grammar learning, switched from being virtually unknown to being a fundamental approach in the last ten years [1]. Thanks to this, we now have available a big number of corpora which we can use to extract statistical information and can help us to understand the underlying structures of the languages. In fact, these data sets are usually annotated and we can use these annotations, together with statistics, in order to spot certain regularities or frequent characteristics of a language. Within the multiple tasks that can be performed with such knowledge, the target of this project is the machine reconstruction of the syntax (parse) of a sentence. We would like to understand which rules generated a sentence, in order to understand which class of sentence we are dealing with. To better explain the task, we can use the noisy channel model [8]: In this model (represented in figure 1), we assume that A has a concept and wants B to receive this concept. In our case this concept is the correct parse for a sentence. B does not receive the full parse, but just the sentence, so he needs to reconstruct the correct parse that A had in mind starting from the observations he gets through the noisy channel (in figure 1, this is represented by the dashes without the parse).